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LONG-TERM  GOALS 

The  long-term  goals  of  this  project  are  to  improve  the  quantitative  accuracy  and  interpretative  utility  of 
official  tropical  cyclone  (TC)  track  forecasts  by  enabling  forecasters  to  successfully  recognize  and 
skillfully  compensate  for  periods  when  numerical  TC  track  forecast  models  are  likely  to  be  making 
highly  erroneous  track  forecasts.  The  conceptual  methodology  for  accomplishing  these  goals  is  the 
Systematic  Approach  to  Tropical  Cyclone  Track  Forecasting  (hereafter  Systematic  Approach) 
conceived  by  Carr  and  Elsberry  (1994). 

OBJECTIVES 

The  specific  objectives  of  this  project  are  to:  (i)  develop  a  prototype  expert  system  based  on  the 
Systematic  Approach;  and  (ii)  demonstrate  the  feasibility  of  such  an  expert  system  for  improving  the 
accuracy  and  meteorological  utility  of  official  tropical  cyclone  track  forecasts.  It  is  emphasized  that  the 
purpose  of  the  expert  system  is  not  to  replace  the  human  forecaster,  but  to  proactively  lead  the 
forecaster  through  a  methodical  process  of  numerical  guidance  evaluation  and  forecast  formulation  that 
produces  consistently  skillful  official  track  forecasts. 

APPROACH 

Following  the  conceptual  framework  of  the  Systematic  Approach  (Fig.  1),  the  expert  system  will  be  a 
series  of  inter-linked  software  modules  that  assist  the  forecaster  to  accomplish  each  of  the  three  main 
processes.  To  field  a  prototype  expert  system  as  expeditiously  as  possible,  software  development 
commenced  with  the  module  that  implements  Phase  II.  Subsequent  versions  will  incorporate  Phases  I 
and  II. 
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Figure  1.  Schematic  of  the  Systematic  Approach  procedural  framework,  including  the  three  main 
decision-making  procesess  and  the  associated  information  resources  and  knowledge  bases. 


WORK  COMPLETED 

A  testable  expert  system  prototype  (Peak  et  al.  2000)  that  implements  Phase  II  (Fig.  1)  was  completed 
in  July  1999  for  real-time  utilization  and  evaluation  at  the  Naval  Postgraduate  School  (NPS).  The 
critical  Model  Traits  Knowledge  Base  for  the  prototype  is  contained  in  Carr  and  Elsberry  (1999, 
2000a,b).  At  the  request  of  JTWC,  NPS  agreed  to  name  the  expert  system  the  Systematic  Approach 
Forecasting  Aid  (SAFA).  A  beta  test  of  the  SAFA  prototype  was  conducted  at  NPS  by  a  team  of 
researchers  and  forecasters  from  23  August  to  1  December  1999,  which  covered  TC  19W  through 
30W.  The  results  of  the  beta-test  are  documented  in  Carr  et  al.  (2000a,b,c).  Based  on  the  results,  a 
number  of  refinements  were  made  to  the  SAFA  software,  and  several  web  browser-based  training 
modules  were  developed  for  forecaster  familiarization  and  proficiency  with  the  initial  operational 
version  of  SAFA  that  was  fielded  in  July  2000. 
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RESULTS 


Key  objectives  of  the  beta  test  by  NPS  were  to:  (i)  validate  the  error  mechanism  frequencies  suggested 
by  previous  research  associated  with  this  project  (Carr  and  Elsberry  1999,  2000a,b,c);  (ii)  determining 
whether  error  mechanisms  could  be  correctly  identified  without  the  benefit  of  hindsight;  and  (iii) 
documenting  that  a  selective  consensus  (SCON)  track  forecast  that  excludes  dynamical  model  tracks 
deemed  to  be  unacceptably  erroneous  is  more  accurate  that  a  non-selective  consensus  (NCON)  forecast 
that  includes  all  available  dynamical  model  track  forecasts. 

With  regard  to  (i)  and  (ii)  above.  Table  1  provides  error  mechanism  totals  summed  over  all  models,  as 
well  as  the  number  of  misses  and  false  alarms  made  by  the  NPS  beta-test  team.  In  general,  the  error 
frequencies  are  similar  to  those  in  Carr  and  Elsberry  (1999)  with  Excessive-Direct  Cyclone  Interaction 
(E-DCI)  being  the  most  frequent  error  mechanism.  Other  frequently  occurring  error  mechanisms  in  the 
developmental  sample  such  as  Excessive-Ridge  Modification  by  a  TC  (E-RMT),  Excessive-Reverse 
Trough  Formation  (E-RTF),  and  Excessive-Response  to  Vertical  wind  Shear  (E-RVS)  also  occurred 
frequently  during  the  beta  test.  These  four  frequently  occurring  error  mechanisms  (E-  DCI,  E-RMT,  E- 
RTF,  E-RVS)  were  generally  recognized  successfully  by  the  NPS  beta-test  team.  That  is,  the  numbers 
of  misses  and  false  alarms  are  significantly  smaller  than  the  number  of  occurrences  of  each  error 
mechanism  (Table  1).  In  the  cases  of  E-DCI  and  E-RMT,  the  false  alarm  rates  are  higher  than  desired. 
However,  several  false  alarms  for  these  mechanisms  were  for  model  track  forecasts  that  were  indeed 
degraded,  but  the  degradation  did  not  result  in  a  72-h  error  greater  than  300  n  mi. 


ERROR  MECHANISM  NAME 

Total  number  of 
Occurrences 

Misses 

False 

Alarms 

Direct  Cyclone  Interaction 

DCI 

29 

1 

7 

Semi-direct  Cyclone  Interaction 

SCI 

SCI  on  Western  TC 

saw 

SCI  on  Eastern  TC 

SCIE 

0 

0 

2 

Indirect  Cyclone  Interaction 

ICI 

ICI  on  Eastern  TC 

ICIE 

1 

0 

0 

ICI  on  Western  TC 

ICIW 

2 

2 

1 

Ridge  Modification  by  TC 

RMT 

8 

1 

3 

Reverse  Trough  Formation 

RTF 

8 

3 

1 

Response  to  Vertical  wind  Shear 

RVS 

8 

1 

0 

Baroclinic  Cyclone  Interaction 

BCI 

4 

4 

0 

Midlatitude  System  Evolutions 

MSE 

Midlatitude  CycloGenesis 

MCG 

2 

0 

4 

Midlatitude  CycloLysis 

MCL 

Midlatitude  AnticycloGenesis 

MAG 

3 

1 

0 

Midlatitude  AnticycloLysis 

MAL 

1 

0 

0 

Monsoon  Gyre-TC  Interaction 

GT1 

3 

3 

0 

Not  discernible  or  explainable 

13 

1 

1 

Table  1.  Total  occurrence  ( column  3)  of  various  error  mechanisms  ( regardless  of  whether  character 
is  excessive  or  insufficient),  misses  by  NPS  beta-test  team  ( column  4),  and  false  alarms  by  NPS  beta- 
test  team  (column  5)  during  the  period  of  the  SAFA  beta-test.  Rows  in  bold  ( italics )  indicate  error 
mechanisms  for  which  a  low  (high)  incidence  of  misses  or  false  alarms  occurred. 
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The  third  objective  of  the  beta  test  was  to  demonstrate  that,  on  average,  a  SCON  track  after  deletion  of 
one  or  more  dynamical  model  tracks  will  have  smaller  errors  than  the  NCON  track.  Figure  2  depicts  a 
case  for  which  SCON  was  a  marked  improvement  over  NCON,  which  illustrates  the  considerable 
value-added  that  can  be  achieved  using  SAFA.  To  assess  the  average  performance  of  SCON  relative  to 
NCON  during  the  SAFA  beta  test,  only  a  relatively  small  sample  is  available:  61,  46,  and  29  forecasts 
at  24,  48,  and  72  h,  respectively.  The  rapid  fall-off  of  the  sample  size  with  increasing  forecast  interval 
indicates  that  TCs  19W  to  30W  were  relatively  short-lived  so  that  often  a  verifying  position  was  not 
available,  especially  at  72  h.  For  the  29  cases  at  72  h,  the  FTEs  for  the  CLIPER,  JTWC,  NCON,  and 
SCON  were  305,  220,  205,  and  200  n  mi,  respectively.  Thus,  both  SCON  and  NCON  forecasts  were  an 
improvement  over  the  official  forecasts,  and  about  100  n  mi  better  than  the  CLIPER,  which  is  then 
considered  as  a  33%  skill. 

As  it  became  evident  that  only  a  small  average  SCON-NCON  difference  was  being  achieved  when 
some  individual  cases  of  marked  improvement  of  the  SCON  track  relative  to  the  NCON  track  had  been 
achieved,  a  mid-test  evaluation  was  made  as  to  the  cases  when  the  SCON  errors  were  greater  than  the 
NCON  errors.  In  seven  of  these  eight  cases,  the  consensus  spread  of  the  dynamical  model  tracks 
relative  to  the  NCON  position  was  less  than  250  n  mi,  which  the  data  of  Elsberry  and  Carr  (2000) 
suggest  may  be  a  threshold  for  a  relatively  tight  cluster. 
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Figure  2.  Screen  capture  of  the  Tracks  Display  portion  of  the  SAFA  screen  showing  a  SCON 
forecast  (average  of  green  “accepted”  dynamical  model  tracks)  that  is  considerably  more  accurate 
than  the  NCON  forecast  (average  of  all  five  tracks ).  The  two  red  tracks  were  rejected  by  the  NPS 
beta  test  team  due  to  recognized  indications  of  Excessive  Reverse  Trough  Formation  (E-RTF)  in  the 
forecast  fields  of  the  Japanese  Global  Spectral  model  (JGSM).  All  forecast  tracks  extend  to  72  h  and 
the  single  letter  labels  correspond  to  the  enlarged  letter  in  model-selection  buttons  in  the  lower  left. 
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Based  on  this  discovery,  the  FTE  statistics  were  recomputed  with  the  SCON  track  set  equal  to  NCON 
whenever  the  consensus  spread  was  less  than  250  n  mi.  For  the  resulting  31  cases  at  72  h,  the  FTEs  for 
CLIPER,  JTWC,  NCON,  and  SCON  were  292,  217,  196,  and  186  n  mi,  respectively.  Of  these  31  cases, 
SCON  was  equal  to  NCON  (i.e.,  no  model  tracks  rejected)  17  times.  In  the  remaining  14  cases,  for 
which  the  NPS  team  attempted  to  add  value  by  not  accepting  NCON,  the  improvement  of  SCON 
relative  to  NCON  at  72  h  increases  to  23  n  mi  (Fig.  3).  Thus,  the  NPS  team  was  able  to  detect  model 
tracks  with  large  errors  such  that  the  average  SCON  track  error  at  72  h  was  about  10%  smaller  than  for 
the  average  NCON  whenever  the  72-h  ensemble  spread  exceeded  250  n  mi. 

TRANSITIONS 

The  results  of  the  SAFA  beta-test  (Carr  et  al.  2000a, b,c)  were  favorably  received  by  the  leadership  of 
JTWC,  thus  warranting  development  (via  SPAWAR  funding)  of  the  SAFA  prototype  into  a  fully 
operational  expert  system.  SAFA  Version  1.0  was  completed  and  installed  at  JTWC  in  early  July  2000. 
As  of  this  writing,  JTWC  is  having  a  potentially  record- setting  year  in  terms  of  track  forecast  accuracy. 
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Figure  3.  Skill  (in  n  mi.)  relative  to  the  CLIPER  forecast  track  errors  (FTE)  listed  along  the  zero 
line  for  a  homogeneous  sample  (number  of  cases  indicated)  from  TC 19W  through  30W  during 
August-December  1999 for  the  Joint  Typhoon  Warning  Center  (JTWC),  a  non-selective  consensus 
(NCON),  and  a  selective  consensus  (SCON)  track  forecast  by  the  NPS  beta-test  team. 
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